Introducing Latency And Delay For Test Or Debug Purposes In A SAN Environment

ABSTRACT

Simulating latency in a network environment. A device sends a latency request. The device receives a latency support confirmation. The device builds an I/O frame. The I/O frame comprises a latency simulating bit and a latency duration. Based on the latency simulating bit and the latency duration, the device holds the I/O frame. The device sends the I/O frame.

BACKGROUND

The present disclosure relates generally to computing systems, and morespecifically, to stress-testing a computer system network.

Due to cost and space limitations, many test environments have smallerscale Storage Area Network (SAN) environments with smaller workloads.Testing includes adding latency and simulating a larger SAN hop count,which is the number of hops needed to get from the sender to thereceiver to more accurately test products. Current methods that exist tointroduce latency and simulate hop count include the addition ofphysical test tools placed in the environment. For example, U.S. Pat.No. 6,922,663, incorporated herein by reference, teaches a networkclient simulation tool that virtualizes multiple clients from a singlenetwork entity. The simulation tool builds LAN frames of protocol stacklevel 2, which represent data originating from multiple clients tosimulate traffic from multiple clients. It also provides a method forinserting the level 2 LAN frames built onto a physical LAN for actualdelivery to a system being tested for simulating real client LAN trafficenvironment. U.S. Pat. No. 8,102,776, incorporated herein by reference,teaches that simulated network traffic may be generated using aspecification of a sequence of frames to be transmitted from the networktesting device These tools can only impact the links they are placed on(limited ports) and the tool costs prevent the wide spread use acrosstest teams and configurations.

BRIEF SUMMARY

According to one embodiment, a method, system, and program product isprovided. A device sends a latency request. The device receives alatency support confirmation. The device builds an I/O frame. The I/Oframe comprises a latency simulating bit and a latency duration. Basedon the latency simulating bit and the latency duration, the device holdsthe I/O frame. The device sends the I/O frame.

According to one embodiment, the holding further comprises calculating,by the device, a latency period based on the latency duration. Thelatency period is a period of time that the I/O frame is held for.

According to one embodiment, the latency period is calculated by thedevice using a statistical distribution.

According to one embodiment, the statistical distribution comprises afunction selected from a group consisting of: a constant value; anexponential probability density function, the latency delay representingthe exponential probability density function by an expected value of thelatency delay representing the 63.2^(nd) percentile; an uniformprobability density function, the latency delay representing the uniformprobability density function by a lower limit and an upper limit of thelatency delay; a Gaussian probability density function, the latencydelay representing the Gaussian probability density function by anarithmetic mean representing the 50^(th) percentile, a standarddeviation of said latency delay, and the number of standard deviationsto be included; a binary probability density function, the latency delayrepresenting the binary probability density function by a first latencyduration and a second latency duration; a Rayleigh probability densityfunction, the latency delay representing the Rayleigh probabilitydensity function by an expected value of the latency delay representingthe 63.2^(nd) percentile and a shape factor of two; and a Weibullprobability density function, the latency delay representing the Weibullprobability density function by an expected value of the latencyduration representing the 63.2^(nd) percentile, a shape factor, and alatency offset.

According to one embodiment, the device is a server or a switch.

According to one embodiment, the device comprises a controller. Thecontroller is configured for the holding of the I/O frame.

According to one embodiment, the I/O frame comprises an I/O command.

According to one embodiment, the sending of the I/O device comprisessending the I/O frame to a second device.

According to one embodiment, the second device is a server or a switch.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The subject matter which is regarded as the invention is particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The foregoing and other features, and advantages ofthe disclosure are apparent from the following detailed descriptiontaken in conjunction with the accompanying drawings in which:

FIG. 1 illustrates a network including a switch connecting two servers,in accordance with an embodiment;

FIG. 2 illustrates a Fabric Log In (FLOGI) frame, in accordance with anembodiment;

FIG. 3 illustrates a payload, including reserve bits, in accordance withan embodiment;

FIG. 4 illustrates Common Service Parameters for Port Log In (PLOGI),Fabric Log In (FLOGI), and Fabric Log In Link Service Accept (FLOGILS_ACC), in accordance with an embodiment;

FIG. 5 illustrates an I/O command frame, including a delay header field,in accordance with an embodiment;

FIG. 6 illustrates choices in latency delay, in accordance with anembodiment;

FIG. 7 illustrates a process for latency simulation in accordance anembodiment;

FIG. 8 illustrates a computer program product to incorporate anembodiment; and

FIG. 9 illustrates a computer system in which an embodiment may bepracticed.

DETAILED DESCRIPTION

In accordance with an embodiment, a method, system, and computer programproduct is provided for stress-testing a computer system network, inparticular, adding latency and simulating SAN hops to the computersystem network. Embodiments described herein are directed to testingnetwork communications. Specific details regarding networking,particularly for Fibre Channel, can be found in “Fibre Channel LinkServices (FC-LS-3) REV 3.0”, published Feb. 21, 2012, incorporatedherein by reference; “Fibre Channel Framing and Signaling-2 (FC-FS-2)Rev 0.00”, published May 7, 2003, incorporated herein by reference;“Fibre Channel Arbitrated Loop (FC-AL-2) Rev 7.0, published Apr. 1,1999, incorporated herein by reference; and “Fibre Channel Framing andSignaling-4 (FC-FS-4) Rev 0.10”, published Apr. 17, 2012, incorporatedherein by reference.

For example, FC-LS-3 teaches the following:

Class 2 service: A service that multiplexes frames at frame boundariesto or from one or more Nx_Ports with acknowledgement provided.

Class 3 service: A service that multiplexes frames at frame boundariesto or from one or more Nx_Ports without acknowledgement.

Data Frame: An FC-4 Device_Data frame, an FC-4 Video_Data frame, or aLink_Data frame

Destination_Identifier (D_ID): The address identifier used to indicatethe targeted destination Nx_Port of the transmitted frame.

Fabric: The entity that interconnects Nx_Ports attached to it and iscapable of routing frames by using the D_ID information in aFrame_Header.

F_Port: An FC_Port within the Fabric that attaches to a PN_Port througha link, and is addressable by the Nx_Ports communicating through thePN_Port attached to the F_Port by the F_Port Controller well-knownaddress (i.e., FFFFFEh)

FC_Port: A port that is capable of transmitting and receiving FibreChannel framesaccording to the FC-0, FC-1, and FC-2 levels of the FibreChannel architecture.

Information Category: The category to which the frame Payload belongs(e.g., Solicited Data, Unsolicited Data, Solicited Control andUnsolicited Control)

Link Control Facility (LCF): A hardware facility that attaches to an endof a link and manages transmission and reception of data.

Name_Identifier: A 64-bit identifier, with a 60-bit value preceded by a4-bit Network_Address_Authority Identifier, used to identify entities inFibre Channel (e.g., Nx_Port, node, F_Port, or Fabric)

Network_Address_Authority (NAA): An organization such as Institute ofElectrical and Electronics Engineers (IEEE) that administers networkaddresses.

Network_Address_Authority (NAA) identifier: A four-bit identifierdefined to indicate a Network_Address_Authority (NAA).

N_Port: An Nx_Port communicating through a PN_Port that is not operatinga Loop Port State Machine, including services operating at well-knownaddresses.

N_Port_ID: An address identifier of an Nx_Port used in the identifysource (S_ID) and destination (D_ID) fields of a frame that is uniquewithin a topology.

NL_Port: An Nx_Port communicating through a PN_Port that is operating aLoop Port State Machine and without the qualifier Public or Private isassumed to be a Public NL_Port.

Nx_Port: An end point for Fibre Channel frame communication (seeFC-FS-3) that is used in this standard to specify behavior of eitherN_Ports or Public NL_Ports that does not specify or constrain thebehavior of Private NL_Ports.

Payload: Contents of the Data_Field of a frame, excluding OptionalHeaders and fill bytes, if present.

PN_Port: An LCF that supports only Nx_Ports.

Public NL_Port: An NL_Port that attempts a Fabric Login.

Private NL_Port: An NL_Port that does not attempt a Fabric Login anddoes not transmit open primitive (OPN).

Sequence: A set of one or more Data frames with a common Sequence_ID(SEQ_ID), transmitted unidirectionally from one Nx_Port to anotherNx_Port with a corresponding response, if applicable, transmitted inresponse to each Data frame.

Sequence_ID (SEQ_ID): An identifier used to identify a Sequence.

One embodiment of a network is described in reference to FIG. 1. FIG. 1illustrates network 100 with servers 102, 112 and intervening switch120. Each server comprises network adapter 104, 114 and storage 106, 116and controller 108, 118. Each network adapter has a transmit module104A, 114A and a receive module 104B, 114B and a buffer 104C, 114C. Eachcontroller 108, 118 has a buffer 109, 119, respectively. Eachcontroller, 108, 118, 124 may also include a random number generator(not shown). Switch 120 has buffer 122 and controller 124.Communications between servers 102, 112 occurs through switch 120, andmay use Gigabet Ethernet (GbEN), Fibre Channel, Fibre Channel overEthernet, Small Computer System Interface (SCSI), Internet ComputerSystem Interface (iSCSI), Infiniband protocols, and the like.

One embodiment of a Fabric Log In (FLOGI) frame is described inreference to FIG. 2. FIG. 2 shows FLOGI frame 200, comprising aStart_of_Frame (SOF) delimiter 201 (4 bytes), Frame Feader 202 (24bytes), Network_Header 203 (16 bytes), Association_Header 204 (32bytes), Device_Header 205 (16, 32, or 64 bytes), a Payload 300 (forexample a FLOGI, Port Login (PLOGI), Discover F_Port (FDISC) or LinkService Accept (LS_ACC) payload) described in FIG. 3, fill bytes 206(0-3 bytes), Cyclical Redundancy Check (CRC) 207 (4 bytes), and an Endof Frame delimiter 208 (4 bytes). The Network_Header 203,Association_Header 204, Device_Header 205 and Fill Bytes 206 may beoptional in the frame. The data field comprises the Network Header 203,association header 204 (32 bytes), device_header 205, payload 300, fillbytes 206 (0-3 bytes). The data field may also comprise an EncapsulatingSecurity Payload (ESP) Header and Trailer, not shown in FIG. 2, which isused for ESP, a mechanism to provide confidentiality, data origin,authentication, and anti-replay protection of Internet Protocol (IP)packets.

The SOF delimiter 201 is an ordered set that immediately precedes theframe content. It marks the beginning of the frame. The Frame Header 202is 24 bytes of information used in Fibre Channel frames and containsinformation such as routing control, destination identifier, originatingexchange, source identifier, etc. Network Header 203, which may also beknown as the Ethernet Header, is an optional header whose presence maybe indicated by a field in the Frame_Header 202.

The Network Header 203 may be used for routing between Fibre Channelnetworks of different Fabric address spaces, or Fibre Channel andnon-Fibre Channel networks. The Association Header 204 may be used toidentify a specific process or group of processes within a nodeassociated with an exchange. When an Nx_Port has indicated during Loginthat an Initial Process Associator is required to communicate with it,the Association Header 204 shall be used by that Nx_Port to identify aspecific Process or group of Processes within a node associated with anExchange. The Device Header 205 may contain FC-4/Upper layer protocolspecific data based on a Type field in the Frame Header 202. If present,the Device Header 205 may be present in either the first Data frame orin all Data frames of a Sequence. Upper level protocol (ULP) types mayalso make use of the Device Header 205. The Device Header 205 may beignored and skipped if not needed. If a Device Header 205 is present fora ULP that does not require it, the related FC-4 level of the FibreChannel protocol may reject the frame. The Payload 300, furtherdescribed in FIG. 3, contains the contents of the data field of a frame,excluding optional headers, 203, 204, 205 and Fill Bytes 206, ifpresent. The Fill Bytes 206 is used to pad out a frame to a wordboundary because of Fibre Channel requirements that all frames end on a4-byte boundary. The CRC 207 may be used to verify the data integrity ofthe data within the frame. The End of Frame (EOF) delimiter 208 isrepresented by an Ordered Set that immediately follows the CRC. The EOFOrdered Set may designate the end of the frame.

One embodiment of a payload is described in reference to FIG. 3. Thepayload 300 may be representative of a FLOGI, PLOGI, FDISC, or LS_ACCpayload. Payload 300 includes Extended Link Services (ELS) Command code301, Common Service Parameters (16 bytes) 400 which is further discussedin FIG. 4, Port_name 302, Node_(—) or Fabric_name 303, Obsolete bytes304, Class 2 Service Parameters 305 (16 bytes), Class 3 ServiceParameters 306 (16 bytes), Auxiliary Parameter Data 307 (16 bytes),Vendor Version Level 308 (16 bytes), Service Availability 309 (8 bytes),Login Extension Data Length 3010, Reserved Bytes 311, ClockSynchronization QoS 312 (8 bytes), and Login Extension Data 313 (ifany).

ELS_Command code 301 may be a 4-byte field (word 0) that identifies, forexample a PLOGI, FLOGI, or LS_ACC. Every extended link service isassigned a code. It defines the bits and bits that follow this field.

The Port_Name 302 may be an eight-byte field (words 5-6) that identifiesan Nx_Port. Each Nx_Port, including Nx_Ports that have Well-knownaddresses, shall provide a Name_Identifier. Nx_Ports that are notassigned to Well-known addresses shall provide a Name_Identifier that isunique within the Fibre Channel interaction space of the Nx_Port. Bits63-60 specify the format of the Name_Identifier.

The Node_Name or Fabric_Name 303 may be an eight-byte field (words 7-8)that labels a Node or Fabric for identification purposes, such asdiagnostics. The Node_Name and Fabric_Name 303 are independent of andunrelated to network addressing. Each Node_Name or Fabric_Name 303 shallbe unique within the Fibre Channel interaction space. Bits 63-60 specifythe format of the name. Node_Name may be applicable to PLOGI, PLOGILS_ACC and FLOGI. Fabric_Name may be applicable to FLOGI LS_ACC.

The Obsolete 304 may be a 16-byte field (words 9-12). This field is notused.

The Class 2 Service Parameters 305 may be a 16-byte field (words 13-16).The Class 3 Service Parameters 306 may be a 16-byte field (words 17-20).These parameters are further described in FIG. 4.

The Auxiliary Parameter Data 307, which may also be known as the DataField Control (DF_CTL), may be a 16-byte field (words 21-24). This fieldspecifies whether the data portion of Payload 300 is used for a non datapurpose. It may also specify the first n number of bytes designated forthe non data purpose.

Vendor Version Level 308 field may be a 16-byte field (words 25-28) andmay specify vendor-specific information. If the Valid Version Level bitin the Common Service Parameters field 400 (word 1, bit 29) is set toone, the Vendor Version Level field 308 contains valid information. Ifthe Valid Version Level bit is set to zero, the Vendor Version Levelfield 308 is not meaningful.

The Services Availability field 309 may be a 8-byte field (word 29-30)that returns information regarding the Fabric's ability to route to thedefined well-known addresses. It is meaningful only for FLOGI LS_ACC.Only bits 10-3 of word 30 are meaningful. Word 29 and bits 31-11 and 2-0of word 30 are reserved.

The Login Extension Data Length 310 may be an 8-byte field (word 31). Ifthe Login Extension Data Length field 310 is non-zero, a Login Extensionfollows the normal payload. The Login Extension Data Length field 310indicates the length of the Login Extension field in words. The PayloadBit located in the Common Service Parameters 400 shall be set to one ifthis field is non-zero.

The Reserved Bytes 311 may be 32 bits, or 4 bytes, per word. In oneembodiment, the Reserved Bytes may include 30 words, for example BitsWord 32-61, thus resulting in 120 bytes total in reserve, for futureuse. The Reserved Bytes may include a Latency Delay Bit (also known asLatency Delay Enabled bit or Latency Simulating Bit) and Maximum Numberof Latency Delayed Frames, 314.

In one embodiment of the invention, the Latency Delay Enabled Bit andthe Maximum Number of Latency Delayed frames, 314, may be located in thefirst byte. The first bit of this byte denotes whether the latency delayis disabled or enabled. For example, disabled may be represented by thebit=0 and enabled may be represented by the bit=1. If the latency delayis enabled (bit=1), then the controllers 108,118,124 know to search theI/O command 500 for delay header information which is then utilized. Ifthe remaining bits of this byte are zero, then the number of frames witha latency delay 314 are unlimited, until the latency delay process isterminated by the user. However, if the remaining bits of this byte arenon-zero, then the number of frames which are latency delayed 314 arelimited to the number denoted by those remaining seven bits, after whichtime the latency-delay process auto-terminates. Additional reservedbytes may be appended to the first byte to add to the number of frameswhich are latency delayed before the latency delay processauto-terminates. Thus, the latency delay process may be enabled via aFLOGI, PLOGI, FDISC, or a LS_ACC.

The Clock Synchronization Quality of Service (QoS) field may be 8-bytes(word 62-63) for syncing the clock for the purposes of the simulateddelay. This insures consistent simulated delays. This field revolvesaround a service, for example a service that resides in a switch, calledthe Clock Synchronization Server. During the login process, this fieldis used to check if the Clock Synchronization Server exists.

The Login Extension Data 313 may be specified by the any number of byteslocated from word 64 and on. This field specifies additional data thatmay be necessary for a login, which extends the size of the login frame.

One embodiment of Common Service Parameters for PLOGI, FLOGI, and FLOGILS_ACC, is described in reference to FIG. 4. FIG. 4 illustrates CommonService Parameters 400, of which FLOGI LS_ACC is a part. Common ServiceParameters 400 may be defined by the Fibre Channel Link Servicesspecification. Common Service Parameters 400 may include Fibre ChannelPhysical Interface (FC-PH) Version 401 (which may be obsolete and notused), Buffer-to-Buffer Credit 402 and Common Features 403. CommonFeatures 403 may include Continuously Increasing Relative Offset 404,Clean Address 404, Multiple N_Port_ID Support 406, Random RelativeOffset 407, Virtual Fabrics Bit 408, Valid Vendor Version Level 409, andMultiple N_Port_ID Assignment 410. FIG. 4 further illustrates the Word411, Bits 412, Default Login Value 413 and the parameter applicabilityfor PLOGI and PLOGI LS_ACC 414, FLOGI 415, AND FLOGI LS_ACC 416. Eachparameter applicability 414, 415, 416 may be use in either Class 2 orClass 3 protocols, or both. Class 2 is an acknowledgement protocol,where every frame has an ACK(nowledgement), which adds overhead. Class 3is unacknowledged transmission of frames, which offers higherperformance due to the absence of ACKs. The use of Class 3 is moreprevalent for data transmission, with Class 2 being used for criticalcommand and control.

The buffer-to-buffer Credit field 402 (word 0, bits 15-0) defines thenumber of buffers available for holding Class 2, or Class 3 framesreceived. An FC_Port tracks Buffer-to-buffer Credit as a single entityfor all frames subject to buffer-to-buffer flow control. Values in theBuffer-to-buffer Credit field 402 are 1 to 32 767. The value 0 isreserved.

Continuously increasing relative offset 404 may comprise a bit, forexample word 1, bit 31. If the continuously increasing relative offsetbit (word 1, bit 31) is set to one, the Nx_Port supplying this parametershall be capable of supporting continuously increasing relative offset,if present, within a Sequence on a frame by frame SEQ_CNT (sequencecount) basis. This bit shall only be applicable to those InformationCategories in which an Nx_Port supports relative offset (i.e., word 2,bits 15-0).

The Clean Address bit 405 (word 1, bit 31) provides an indication to anNx_Port as to whether the address it was assigned by the Fabric had beenpreviously used by another device within a Resource_Allocation_Timeoutvalue (R_A_TOV). If this bit is set to zero, the assigned address may ormay not have been used by a previous device within R_A_TOV. If this bitis set to one, the assigned address has not been used by any otherdevice within R_A_TOV, or has been assigned to the current device for aprevious FLOGI and not been changed within R_A_TOV. This bit is onlymeaningful in the FLOGI LS_ACC, it is not meaningful in the FLOGIrequest.

The Multiple N_Port_ID Support bit 406 (word 1, bit 31) shall be set toone to indicate that the PN_Port supplying this parameter is capable ofrequesting multiple N_Port_IDs using the FDISC ELS. The N_Port_IDSupport bit shall be set to zero to indicate that the PN_Port supplyingthis parameter is not capable of requesting additional N_Port_IDs. Thisbit is only meaningful in the FLOGI request, it is not meaningful in theFLOGI LS_ACC.

The Random relative offset 407 (word 1, bit 30) indicates that theNx_Port supplying this parameter shall be capable of supporting randomrelative offset values, if present. Random values may increase,decrease, or otherwise fluctuate within a Sequence. This bit shall onlybe applicable to those Information Categories in which an Nx_Portsupports relative offset (i.e., word 3, bits 15-0).

The Virtual Fabrics bit 408 (word 1, bit 30) indicates support forVirtual fabrics. Virtual fabrics, for example, may include VirtualCluster Switching (VCS) Fabric technology which is a Layer 2 Ethernettechnology designed to improve network utilization, maximize applicationavailability, increase scalability, and dramatically simplify thenetwork architecture in virtualized data centers. It comprises threepillars of innovation technology: Ethernet fabrics, DistributedIntelligence, and Logical Chassis. VCS Fabric technology is designed toincorporate a set of Dynamic Services for the highest level offunctionality and investment protection for data centers, making it acore building block for virtualizing data center networks.

The Valid Vendor Version level 409 (world 1, bit 29), in PLOGI, PLOGILS_ACC, and FLOGI, may be set to one or zero. If the Valid VendorVersion Level bit 409 is set to one, the Vendor Version Level 308 (words25 through 28 in FIG. 3) contains valid information. If it is set tozero, the Vendor Version Level field 409 is not meaningful.

The Multiple N_Port_ID Assignment 410 (word 1, 29) may be a one or azero. When the Multiple N_Port_ID Support bit 406 in the FLOGI requestis one, the Multiple N_Port_ID Assignment bit 410 shall be set to one ifthe F_Port supplying this parameter is capable of assigning multipleN_Port_IDs to the attached PN_Port using the FDISC ELS. The MultipleN_Port_ID Assignment bit 410 shall be set to zero when the MultipleN_Port_ID Support bit 407 in the FLOGI request is zero or to indicatethat the F_Port is not capable of assigning multiple N_Port_IDs to theattached PN_Port when the Multiple N_Port_ID Support bit 407 in theFLOGI request is one. This bit is only meaningful in the FLOGI LS_ACC,it is not meaningful in the FLOGI request.

One embodiment of an Input/Output (I/O) Command is described inreference to FIG. 5. FIG. 5 illustrates I/O Command 500 which includes aDelay Header field 506 of 4 bytes. Additional components of I/O Command500 includes Start_of_Frame delimiter 501 (4 bytes), Frame Header 502(24 bytes), Network Header 503 (16 bytes), Association header 504 (32bytes), Device Header 505 (16, 32, or 64 bytes), Delay Header 506 (4bytes), Payload 507, Fill Bytes 508 (0-3 bytes), cyclical redundancychecks (CRC) 509 (4 bytes), and End of Frame Delimiter 510 (4 bytes).The Network Header 503, Association Header 504, Device Header 505, DelayHeader 506, and Fill Bytes 508 may be optional. The Start of FrameDelimiter 510, Frame Header 502, Network Header 503, Association Header504, Device Header 505, Payload, Fill Bytes, CRC 509, and End of Framedelimiter 501 fields have been previously described with respect toFIGS. 2 and 3, above.

The Delay Header 506 may comprise a Time Qualifier Bit 511, an ID ofChoice of Probability Density Function 512, a First Parameter 513,Second Parameter 514, and a Third Parameter 515. The Leading TimeQualifier Bit 511 may be located in the first byte of the Delay Header506 and may be a 0 value if the units of time (the time used for thedelay) is in microseconds or a 1 value if the units of time is inmilliseconds. The ID of the Choice of Probability Density Function 512may be located in the last 7 bits of the first byte of the Delay Header506, and is described further in FIG. 6. The First Parameter 515 whichmay be located in the second byte of the Delay Header 506, the SecondParameter 516 which may be located in the third byte of the Delay Header506, and the Third Parameter which may be located in the fourth byte ofthe Delay Header 506, contain information as further described in FIG.6. In one embodiment, this delay header information 509 may be includedin the FLOGI (Fabric Log-In), PLOGI (Port Log-In), FDISC (Discovery ofF_Port Service Parameters), and LS_ACC (Link Service Accept) ReserveBytes 310 described in FIG. 3, and may apply to all commands, such ascontrol commands, and not just limited to data commands such as I/OCommand 500.

FIG. 6 illustrates possible values for the ID of Choice of ProbabilityDensity Function 512, the First Parameter 513, the Second Parameter 514,and the Third Parameter 515, in accordance with an embodiment. TheChoice of Probability Density Function 512 may be a constant latencyduration, an exponentially distributed latency, a uniformly distributedlatency duration, a Gaussian distributed latency duration, a binarydistributed latency duration, a Rayleigh distributed latency duration,and a Weibull distributed latency duration. Exponential, uniform,Gaussian, Rayleigh, and Weibull probability density functions are wellknown in the art. The ID of Choice of Probability Density Function 512column describes the bit values used to identify the type of Choice ofProbability Density Function of Latency Duration 601. The FirstParameter 515, Second Parameter 516, and Third Parameter 517 columnscontain a description of the type of values to be stored in eachrespective field.

The Choice of Probability Density Function of Latency Duration describesmultiple durations. The constant latency duration consists of a singlespecified latency duration. The exponentially distributed latencyduration consists of the expected value of latency duration (63.2^(nd)percentile). The uniformly distributed latency duration includes both aminimum and a maximum latency duration. The Gaussian distributed latencyduration includes the arithmetic mean (average) value of latencyduration (50 percentile), standard deviation of latency duration, andnumber of standard deviations. The binary distributed latency durationincludes a first and a second value of latency duration. The Rayleighdistributed latency duration includes the expected value of latencyduration (63.2^(nd) percentile) and a shape parameter of two. TheWeibull distributed latency duration includes the expected value oflatency duration (63.2^(nd) percentile), a shape parameter, and alatency duration offset, which may be zero.

The exponential probability density function is shown in equation[1](shown below), where C is the characteristic latency duration and C=x at63.2%, exp is the exponential function. Controllers 108, 118, and 124may implement equations [1-4] (shown below), as well as the uniform andbinary distributed latency durations, in software, for example opensource software.

f(x)=(1/C)*exp(−x/C)  equation[1]

The Gaussian probability density function is shown in equation[2] (shownbelow), where U is average value of the latency duration and U=x at 50%,and S is the standard deviation. Since the Gaussian probability densityfunction extends from minus infinity to plus infinity, the domain orinput to the Gaussian probability function is limited to n standarddeviations, where n is user defined.

f(x)={1/[S*√(2π)]}*exp[−(x−U)²/(2S ²)]  equation[2]

The Weibull probability density function is shown in equation[3] (shownbelow), where B is the shape parameter, also know as the Weibull slope,and Xo is the latency duration offset, also known as the locationparameter or minimum value of x. Xo may be set to zero.

f(x)=[B/(C−Xo)]*[(x−Xo)/(C−Xo)]^((B−1))*exp{−[(x−Xo)/(C−Xo)]B}  equation[3]

The Rayleigh probability density function is shown in equation[4] (shownbelow), which is equation[3] with the special case of B=2 and Xo=0.

f(x)=[B/C]*[x/C]*exp{−[x/C] ²}  equation[4]

An exponential probability density function is a special case of theWeibull probability density function, where the shape parameter is unity(B=1) and the latency duration offset Xo is zero. The Rayleighprobability density function is a special case of the Weibullprobability density function, where the shape parameter is two (B=2) andthe latency duration offset Xo is zero. The Weibull probability densityfunction with a shape parameter of B=3.5 mimics the Gaussian probabilityfunction. The binary probability density function is a discreteprobability density function, one equivalent to flipping a coin, whereheads gives the first value of latency duration, and tails gives thesecond value of latency duration. The constant latency duration isequivalent to these special cases: (a) the Gaussian probability densityfunction with a zero standard deviation, (b) a binary probabilitydensity function where the first and second values of latency durationare equal, or (c) a uniform probability density function where theminimum and maximum latency durations are equal.

One embodiment of an implementation is that of the intelligentcontroller in a server, such as servers 102, 112. The location of thelatency buffer is in the controller, such as buffer 109, 119 incontroller 108, 118. The intelligent controller (a) calculates thelatency duration, for example per the Probability Density Functiondescribed in FIG. 6, using any necessary parameters and (b) holds theframe in its buffer until the latency duration has expired before (c)sending the frame to the network adapter, such as network adapter 104,so that the frame can be transmitted to the receiving network adapter,such as network adapter 114 or a network adapter in a receiving switch.No delay header field may be needed in the header, such as the delayheader of I/O Command 500, when the frame is transmitted to thereceiving network adapter, because the entire latency activity isinvisible to the receiving network adapter.

An alternative embodiment is an intelligent controller located in aswitch, such as switch 120. The intelligent controller, such ascontroller 124, (a) calculates the latency duration, for example per theProbability Density Function described in FIG. 6, using any necessaryparameters and (b) holds the frame in its buffer, such as buffer 122,until the latency duration has expired before (c) sending the frame tothe receiving network adapter, such as network adapter 114 or a networkadapter in a receiving switch. No delay header field may be needed inthe header, such as the delay header of I/O Command 500, when the frameis transmitted to the receiving network adapter because the entirelatency activity is invisible to the receiving network adapter.

Yet another alternative embodiment is that of an intelligent transmitterin network adapter, such as transmitter 104A in network adapter 104 or atransmitter in a transmitting switch. The intelligent Transmitter in thesending network adapter (a) calculates the latency duration, for exampleper the Probability Density Function described in FIG. 6, using anynecessary parameters and (b) puts the latency duration in the delayheader as a constant latency duration The receiving network adapter,such as network adapter 114 or a network adapter in a receiving switch,holds the frame in its buffer, such as buffer 114C or a buffer in thereceiving switch, until the latency duration has expired and only then(d) declares the frame to be received.

Yet another alternative embodiment is that of an Intelligent Receiver inNetwork Adapter, such as receiver 114B in network adapter 114 withbuffer 114C or a receiver in a receiving switch. Transmitter, such astransmitter 104A or a transmitter in a transmitting switch, sendsinformation regarding latency duration, for example the delay header 506in I/O frame 500, to a receiver, such as receiver 114B or a receiver ina receiving switch. The delay header 506 in I/O frame 500 may containthe ID of Choice of Probability Density Function 512, the FirstParameter 513, the Second Parameter 514, and the Third Parameter 515 asdescribed in FIG. 5 and FIG. 6. The intelligent receiver (a) puts theI/O frame into its own buffer, such as buffer 114C or a buffer in thereceiving switch, (b) reads the delay header, (c) calculates the latencyduration, and (d) holds the frame in its own buffer until the calculatedlatency duration expires, and only then (e) declares the frame to bereceived.

One embodiment of a process for latency simulation is described inreference to FIG. 7. A first device, for example a server, may issue aninitial I/O request, which includes an I/O command to which the latencysimulation or delay will be applied, for example to all writes, to allreads, or both. The first device, which may be for example a server or aswitch, may send out a latency request to a second device, which may befor example a server or a switch, 701. This latency request is used todetermine if the second device supports latency simulation. The seconddevice receives the latency request, and if it supports latencysimulation, it will send a latency support confirmation back to thefirst device, 702. After receipt of a latency support confirmation, thefirst device may build an I/O frame based on an initial I/O request tosimulate latency 703. The I/O frame may be in the format described inFIG. 5 and may include a latency simulating bit and a maximum number oflatency delayed frames field in the payload as described in FIG. 3. Inone embodiment, the latency simulating bit in the I/O frame may bechecked to make sure that latency simulation is enabled for thatparticular I/O frame before the calculating and holding (704 and 705)steps are performed.

In one embodiment, the first device performs steps 704, 705, and 706. Inthis embodiment, a delay header as previously described above in FIG. 5is not required; however it may be optionally included when the firstdevice builds the I/O frame 703. Based on the I/O frame, a latencyperiod is calculated 704 by the first device, where the latency periodmay depend on the choice of probability density function as described inFIG. 6. Subsequently, the I/O frame is held for the calculated latencyperiod 705 by the first device. After the holding 705, the I/O frame issent 706 by the first device to the second device.

In one embodiment, the second device performs steps 704, 705, and 706.In this embodiment, a delay header as previously described above in FIG.5 may be included when the first device builds the I/O frame 703. Thefirst device sends the I/O frame to the second device for processing,not shown in the figure. Based on the I/O frame, a latency period iscalculated 704 by the second device, where the latency period may dependon the choice of probability density function as described in FIG. 6.Subsequently, the I/O frame is held for the calculated latency period705 by the second device. After the holding 705, the I/O frame is senton 706 by the second device, where it is either released as being areceived I/O frame, or sent on to another device in the network.

In one embodiment, the first device performs step 704 and the seconddevice performs steps 705, and 706. In this embodiment, a delay headeras previously described above in FIG. 5 may be included when the firstdevice builds the I/O frame 703. Based on the I/O frame, a latencyperiod is calculated 704 by the first device, where the latency periodmay depend on the choice of probability density function as described inFIG. 6. The first device sends the I/O frame to the second device forprocessing, not shown in the figure. Subsequently, the I/O frame is heldfor the calculated latency period 705 by the second device. After theholding 705, the I/O frame is sent on 706 by the second device, where itis either released as being a received I/O frame, or sent on to anotherdevice in the network.

In one embodiment, the calculating step 704, whether by the first deviceor the second device, includes checking to see if there is a constantlatency duration in the delay header in the I/O frame. If there is, noadditional calculation is performed, and the process moves on to thenext step.

In one embodiment, based on the maximum number of latency delayed framesfield in the payload of the I/O frame, a subsequent number of frameswill also be held based on the same parameters of the I/O frame.

The above creates a stress-test condition which simulates customerinstallations and workloads without the need of full-scale glass-houseinstallations.

As will be appreciated by one skilled in the art, the embodiments may beembodied as a system, method or computer program product. Accordingly,the embodiments may take the form of an entirely hardware embodiment, anentirely software embodiment (including firmware, resident software,micro-code, etc.) or an embodiment combining software and hardwareaspects that may all generally be referred to herein as a “circuit,”“module” or “system.” Furthermore, the embodiment may take the form of acomputer program product embodied in any tangible medium of expressionhaving computer usable program code embodied in the medium.

One example of a computer program product incorporating one or moreaspects of an embodiment is described with reference to FIG. 8. Acomputer program product 800 includes, for instance, one or morecomputer usable media 802 to store computer readable program code meansor logic 804 thereon to provide and facilitate one or more aspects of anembodiment. Any combination of one or more computer usable or computerreadable medium(s) may be utilized. The computer-usable orcomputer-readable medium may be, for example but not limited to, anelectronic, magnetic, optical, infrared, or semiconductor system,apparatus, or device. More specific examples (a non-exhaustive list) ofthe computer-readable medium would include the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a portable compact disc read-only memory (CDROM), anoptical storage device, or a magnetic storage device. In the context ofthis document, a computer-usable or computer-readable medium may be anystorage medium that can contain or store the program for use by or inconnection with the instruction execution system, apparatus, or device.

Computer program code for carrying out operations of the embodiment maybe written in any combination of one or more programming languages,including an object oriented programming language such as Java,Smalltalk, C++ or the like and conventional procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The program code may execute entirely on the user's computer,partly on the user's computer, as a stand-alone software package, partlyon the user's computer and partly on a remote computer or entirely onthe remote computer or server. In the latter scenario, the remotecomputer may be connected to the user's computer through any type ofnetwork, including a local area network (LAN) or a wide area network(WAN), or the connection may be made to an external computer (forexample, through the Internet using an Internet Service Provider).

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions which execute on the computer or other programmableapparatus provide processes for implementing the functions/actsspecified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments. In this regard, each block in the flowchart or blockdiagrams may represent a module, segment, or portion of code, whichcomprises one or more executable instructions for implementing thespecified logical function(s). It should also be noted that, in somealternative implementations, the functions noted in the block may occurout of the order noted in the figures. For example, two blocks shown insuccession may, in fact, be executed substantially concurrently, or theblocks may sometimes be executed in the reverse order, depending uponthe functionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts, or combinations of special purpose hardware andcomputer instructions.

FIG. 9 illustrates an embodiment of a workstation, server hardwaresystem, in which an embodiment may be practiced. The system comprises acomputer system 901, such as a personal computer, a workstation, aserver, a storage device, or host, including optional peripheraldevices. The computer system 901 includes one or more processors 906 anda bus employed to connect and enable communication between theprocessor(s) 906 and the other components of the computer system 901 inaccordance with known techniques. The bus connects the processor 906 tomemory 905 and long-term storage 907 which can include a hard drive(including any of magnetic media, CD, DVD and Flash Memory for example)or a tape drive for example. The computer system 901 might also includea user interface adapter, which connects the microprocessor 1006 via thebus to one or more interface devices, such as a keyboard 904, mouse 903,a printer/scanner 910 and/or other interface devices, which can be anyuser interface device, such as a touch sensitive screen, digitized entrypad, etc. The bus also connects a display device 902, such as an LCDscreen or monitor, to the microprocessor 906 via a display adapter.

The computer system 901 may communicate with other computers or networksof computers by way of a network adapter 913, for example a networkinterface controller (NIC), capable of communicating 908 with a network909. For example, network adapters may include communications channels,token ring, Ethernet or modems. The computer system 901 may also includea controller, not shown, connected to the network adapter.Alternatively, the computer system 901 may communicate using a wirelessinterface, such as a CDPD (cellular digital packet data) card. Thecomputer system 901 may be associated with such other computers in aLocal Area Network (LAN), VLAN, or a Wide Area Network (WAN), or thecomputer system 901 may be a client in a client/server arrangement withanother computer, etc. All of these configurations, as well as theappropriate communications hardware and software, are known in the art.

Software programming code which embodies an embodiment may be typicallyaccessed by the processor 906 from long-term storage media 907. Thesoftware programming code may be embodied on any of a variety of knownmedia for use with a data processing system, as previously describedabove with reference to FIG. 8. The code may be distributed on suchmedia, or may be distributed to users from the memory or storage of onecomputer system over a network to other computer systems.

Alternatively, the programming code 911 may be embodied in the memory905, and accessed by the processor 906 using the processor bus. Suchprogramming code may include an operating system which controls thefunction and interaction of the various computer components and one ormore application programs 912. Program code may be normally paged fromstorage media 907 to memory 905 where it may be available for processingby the processor 906. The techniques and methods for embodying softwareprogramming code in memory, on physical media, and/or distributingsoftware code via networks are well known and will not be furtherdiscussed herein. The computer program product medium may be typicallyreadable by a processing circuit preferably in a computer system forexecution by the processing circuit.

The flow diagrams depicted herein are just examples. There may be manyvariations to these diagrams or the steps (or operations) describedtherein without departing from the spirit of the embodiment. Forinstance, the steps may be performed in a differing order, or steps maybe added, deleted or modified. All of these variations are considered apart of the claimed embodiment.

While the preferred embodiment has been described, it will be understoodthat those skilled in the art, both now and in the future, may makevarious improvements and enhancements which fall within the scope of theclaims which follow.

What is claimed is:
 1. A method for simulating latency, said methodcomprising: sending, by a device, a latency request; receiving, by saiddevice, a latency support confirmation; building, by said device, an I/Oframe, said I/O frame comprising a latency simulating bit and a latencyduration; based on said latency simulating bit and said latencyduration, holding, by said device, said I/O frame; and sending, by saiddevice, said I/O frame.
 2. The method according to claim 1, wherein saidholding further comprises calculating, by said device, a latency periodbased on said latency duration, said latency period being a period oftime that said I/O frame is held for.
 3. The method according to claim2, wherein said latency period is calculated by said device using astatistical distribution.
 4. The method according to claim 3, whereinsaid statistical distribution comprises a function selected from a groupconsisting of: a constant value; an exponential probability densityfunction, said latency delay representing said exponential probabilitydensity function by an expected value of said latency delay representingthe 63.2^(nd) percentile; an uniform probability density function, saidlatency delay representing said uniform probability density function bya lower limit and an upper limit of said latency delay; a Gaussianprobability density function, said latency delay representing saidGaussian probability density function by an arithmetic mean representingthe 50^(th) percentile, a standard deviation of said latency delay, andthe number of standard deviations to be included; a binary probabilitydensity function, said latency delay representing said binaryprobability density function by a first latency duration and a secondlatency duration; a Rayleigh probability density function, said latencydelay representing said Rayleigh probability density function by anexpected value of said latency delay representing the 63.2^(nd)percentile and a shape factor of two; and a Weibull probability densityfunction, said latency delay representing said Weibull probabilitydensity function by an expected value of said latency durationrepresenting the 63.2^(nd) percentile, a shape factor, and a latencyoffset.
 5. The method according to claim 1, wherein said device is aserver or a switch.
 6. The method according to claim 1, wherein saiddevice comprises a controller, said controller configured for saidholding of said I/O frame.
 7. The method according to claim 1, whereinsaid I/O frame comprises an I/O command.
 8. The method according toclaim 1, wherein said sending of said I/O frame comprises sending saidI/O frame to a second device.
 9. The method according to claim 8,wherein said second device is a server or a switch.
 10. A computersystem comprising: a memory; a processor in communication with saidmemory, said processor comprising an instruction fetching unit forfetching instructions from memory and one or more execution units forexecuting fetched instructions; wherein said computer system is capableof performing a method comprising: sending a latency request; receivinga latency support confirmation; building an I/O frame, said I/O framecomprising a latency simulating bit and a latency duration; based onsaid latency simulating bit and said latency duration, holding said I/Oframe; and sending said I/O frame.
 11. The computer system according toclaim 10, wherein said holding further comprises calculating, a latencyperiod based on said latency duration, said latency period being aperiod of time that said I/O frame is held for.
 12. The computer systemaccording to claim 11, wherein said latency period is calculated using astatistical distribution.
 13. The computer system according to claim 12,wherein said statistical distribution comprises a function selected froma group consisting of: a constant value; an exponential probabilitydensity function, said latency delay representing said exponentialprobability density function by an expected value of said latency delayrepresenting the 63.2^(nd) percentile; an uniform probability densityfunction, said latency delay representing said uniform probabilitydensity function by a lower limit and an upper limit of said latencydelay; a Gaussian probability density function, said latency delayrepresenting said Gaussian probability density function by an arithmeticmean representing the 50^(th) percentile, a standard deviation of saidlatency delay, and the number of standard deviations to be included; abinary probability density function, said latency delay representingsaid binary probability density function by a first latency duration anda second latency duration; a Rayleigh probability density function, saidlatency delay representing said Rayleigh probability density function byan expected value of said latency delay representing the 63.2^(nd)percentile and a shape factor of two; and a Weibull probability densityfunction, said latency delay representing said Weibull probabilitydensity function by an expected value of said latency durationrepresenting the 63.2^(nd) percentile, a shape factor, and a latencyoffset.
 14. The computer system according to claim 10, wherein said I/Oframe comprises an I/O command.
 15. The computer system according toclaim 10, wherein said sending of said I/O frame comprises sending saidI/O frame to a second device.
 16. A computer program product, thecomputer program product comprising: a storage medium readable by aprocessing circuit and storing instructions for execution by theprocessing circuit for performing a method comprising: sending a latencyrequest; receiving a latency support confirmation; building an I/Oframe, said I/O frame comprising a latency simulating bit and a latencyduration; based on said latency simulating bit and said latencyduration, holding said I/O frame; and sending said I/O frame.
 17. Thecomputer program product according to claim 16, wherein said holdingfurther comprises calculating, a latency period based on said latencyduration, said latency period being a period of time that said I/O frameis held for.
 18. The computer program product according to claim 17,wherein said latency period is calculated using a statisticaldistribution.
 19. The computer program product according to claim 16,wherein said statistical distribution comprises a function selected froma group consisting of: a constant value; an exponential probabilitydensity function, said latency delay representing said exponentialprobability density function by an expected value of said latency delayrepresenting the 63.2^(nd) percentile; an uniform probability densityfunction, said latency delay representing said uniform probabilitydensity function by a lower limit and an upper limit of said latencydelay; a Gaussian probability density function, said latency delayrepresenting said Gaussian probability density function by an arithmeticmean representing the 50^(th) percentile, a standard deviation of saidlatency delay, and the number of standard deviations to be included; abinary probability density function, said latency delay representingsaid binary probability density function by a first latency duration anda second latency duration; a Rayleigh probability density function, saidlatency delay representing said Rayleigh probability density function byan expected value of said latency delay representing the 63.2^(nd)percentile and a shape factor of two; and a Weibull probability densityfunction, said latency delay representing said Weibull probabilitydensity function by an expected value of said latency durationrepresenting the 63.2^(nd) percentile, a shape factor, and a latencyoffset.
 20. The computer program product according to claim 16, whereinsaid sending of said I/O frame comprises sending said I/O frame to asecond device.